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ABSTRACT 

The primary objective of the Title I Reading Project 
was to improve the reading skills of sixth- and seventh- grade 
underachievers. Treatment consisted of individualized programs based 
upon diagnostic testing in reading* Project classes in nine ana 11 
schools at the sixth- and seventh-grade levels, respectively were 
evaluated. Selected pupils were randomly assigned to experimental and 
control groups within each school. Analysis of the pretest scores 
revealed that attrition had operated to somewhat bias the samples by 
the end of the study. These biases tended to favor the experimental 
groups* The results of the study indicated that project children did 
no better or worse on the criterion reading skills tests than did 
control students. Attitudes of project and control pupils were also 
similar. It was felt that there was a possibility that the difficulty 
level of the criterion test was inappropriate for these groups. To 
test this possibility, and the possibility that the program had 
delayed benefits, a followup study of the same groups was planned to 
be completed during the 1972-73 school year. Tables and appendixes 
are included. (Author/AW) 
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ABSTRACT . 



The primary objective of the Title I Reading Project was to improve the 
reading skills of sixth- and seventh-grade pupils who fell within 
"normal" IQ ranges but who were identified as reading below their potential. 

Treatment consisted of individualized programs based upon diagnostic test- 
ing in reading. Each student attended a project class one period a day. 
Project teachers all met at least minimum requirements in reading instruc- 
tion. 

Project classes in nine and eleven schools at the sixth- and seventh-grade 
levels respectively were evaluated. The Research Department applied semi- 
empirical criteria in selecting a parent population. These pupils were 
reading below their potential on the basis of available IQ and reading test 
scores. Pupils in this population were randomly assigned to experimental 
and control groups within each school according to "quota" requirements. 

Analysis of pretest scores revealed that attrition had operated to somewhat 
bias the samples by the end of the study. These biases, however, tended to 
favor the experimental groups. 

The results of the study indicated that project children did no better or 
worse on the criterion reading skills’ tests than did control students. 
Attitudes of project and control pupils were also similar. 

There Is a possibility that the level of difficulty of the criterion test 
was inappropriate for these groups. To. test this possibility, and the 
possibility that the program had "delayed" benefits, a follow-up study of 
the same groups -will be done in the early Fall of 1971. The follow-up 
study will be completed in time to assist administrative decision-making 
regarding uses of Title I funds during the 1972-73 school year. 
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TITLE I READING 



The Title I Reading Project was designed to deal with sixth- and seventh- 
year students who are reading below their potential and who may, therefore, 
have difficulties adjusting to the departmentalization of the junior high 
setting. 



Selection of Participants 

Defining who should participate in the project was an important aspect of 
sample selection. Not all students read at grade level. A high IQ 
student, for instance, might be expected to read at an eighth— or even 
ninth— year level. The reverse may be true for lower’ IQ students . To take 
a sixth-year reading level a s the "potential" for all sixth-year students 
is patently false. 

A better estimate of this potential is the mean reading level for sixth 
graders at each IQ level, if a student in the sixth grade is reading at 
,a lower level than the average of all other sixth-year students who have 
the same IQ, then he may be said to be reading "below his potential." 

It is this definition that was used in selecting the sample for the 
Title I Reading Project* 

Articulation cards were gathered on all students entering the sixth and 
seventh grades in Broward County. These cards provided scores obtained 
at the end of the fifth grade on the Otis Short Form Intelligence Test 
and the California Test of Basic Skills . For each grade regression pro^ 
cedures were used to estimate a reading grade equivalent score for each 
IQ level from 80 to 108. (These limits were decided upon by the project 
directors. It was fait* for instance, that while a student with a 75 IQ 
may be reading below his potential, a substantially different approach 
would be needed to help correct his difficulty. This project was designed 
to help the underachiever with a relatively "normal" potential.) 

Table 1 presents the estimates of mean reading levels for each IQ for ' 
the sixth and seventh graders in Broward County junior high schools. 

What the table indicates is that a student going into the sixth grade with 
an IQ of 87 would be predicted to read -on the 3.6 level if he were average 
for his IQ group . 

The realities of operating in an ongoing school system ‘added a number of 
further constraints to the selection process. These constraints made it 
necessary to select the range of reading deficiency used in such a way 
that there would be an app ro priate number of eligible students at each 
school to fill the classes of the previously assigned project teacher (s) • 

At the same time, an adequate number of students, also meeting the criteria, 
had to be left for assignment to the control group. 
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Table 1 



Mean Reading Level for Each IQ 



IQ 


6th Grade 


7th Grade 


80 


2.7 


3.5 


81 


2.8 


3.6 


82 


2.9 


3.8 


83 


3.1 


3.9 


84 


3.2 


4.0 


85 


3.5 


4.2 


86 


3.5 


4.3 


87 


3.6 


4.5 


88 . 


3.7 


4.6 


89 


3.9 


4.8 


90 


4.0 


4.9 


91 


4.1 


5.0 


92 


4.3 


5.2 


93 


4.4 


5.3 


94 


4.5 


5.5 


95 


4.7 


5.6 


96 


4.8 


5.8 


97 


4.9 


5.9 


98 


5.1 


6.0 


99 


5.2 


6.2 


100 


5.3 


6.3 


101 


5.5 


6.5 


102 


5.6 


6.6 


103 


5.7 


6.8 


104 


5.9 


6.9 


105 


6.0 


7.1 


106 


6.1 


7.2 


107 


6.3 


7.3 


108 


6.4 


7.5 



The following range of reading levels was found to fulfill the needs of 
the project centers: the sample consisted of students who were between 

.0.5 years and 2.5 years behind the mean reading grade equivalent for their 
IQ (within the 80 to 108 IQ range) . . 

Table 2 gives the Reading Grade Level range for each IQ for the sixth and 
seventh grades. For example, the table indicates that students going into 
the seventh grade with an IQ of 95 must have a reading level of between 
3.1 .and 5.1 years in order to be eligible for the program. 

From the list of students meeting this criteria in each school, enough 
students were randomly chosen to fill the required number of reading classes 
to a maximum of 80% of capacity. Approximately the same number of students 
were randomly assigned to control groups. 

2 
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Table 2 



Reading Grade Level Range for Each IQ 



IQ 


6th 

Lower Limit 


Grade 

Upper Limit 


7th 

Lower Limit 


Grade 

Upper Limit 


80 


1.0 


2.2 


1.0 


3.0 


81 


1.0 


2.3 


1.1 


3.1 


82 


1.0 


2.4 


1.3 


3.3 


83 


1.0 


2.6 


1.4 


3.4 


84 . 


1,0 


2.7 


1.5 


3.5 


85 


1.0 


3.0 


1.7 


3.7 


86 


1,0 


3.0 


1.8 


3.8 


87 


i.i 


3.1 


2.0 * 


4.0 


88 


1.2 


3.2 


2.1 


4.1 


-89 . 


1.4 


3.4 


2.3 


4.3 


90 


1.5 


3.5 


2.4 


4.4 


91 


1.6 


3.6 


2.5 


4.5 


92 


1.8 


3.8 - 


2.7 


4.7 


93 


1.9 


3.9 


2.8 


4.8 


* 94 


2.0 


4.0 


3.0 


5.0 


95 


2.2 


4.2 


3.1 


5.1 


96 


2.3 


4.3 


3.3 


5.3 


97 


2.4 


4.4 


3.4 


5.4 


98 


2.6 


4.6 


3.5 


5.5 


99 


2.7 


4.7 


3.7 


5.7 


100 


2.8 


4.8 


3.8 


5.8 


101 


3.0 


5.0 


4.0 


6.0 


102 


3.1 


5.1 


' 4.1 


6.1 


103 


3.2 


5.2 


4.3 


6.3 


104 


3.4 


5.4 


4.4 


6.4 


105 


3.5 


5.5 


4.6 


6.6 


106 


3.6 


5.6 


4.7 


6.7 


107 


3.7 


5.7 


4.8 


6.8 


108 


3.9 


5.9 


5.0 


7.0 



Placement of the remaining 20% was left up to the principal’s discretion. 
In about one— third of the schools the principal- was provided with a list 
of students who met the criteria but were simply not chosen in the 
random sample for either the experimental or control group. If no list 
was provided, the classes could be filled with students new to the county 
as long as they were tested and found to meet the criteria. Students 
assigned to project classes by the principal were not used in the evalua- 
tion. In no case were control students to be assigned to the experimental 
condition. 

The actual assignment of students to particular project classes was left 
up to the junior high principals themselves in accordance with the needs 
of their schedules. •• 



Sampling Procedures 



Assignment to Treatment Groups 

The most important aspect of the sampling procedures from a research 
standpoint was the random assignment of pupils to treatment groups 
within schools. The number of experimental classes to be filled and 
the number of eligible students at each school imposed some practical 
restraints upon assignment of equal numbers to both treatment condi- 
tions. No attempt was made to stratify students by pretest scores 
in the randomization procedures. Computer— generated random numbers 
were utilized in assigning pupils. Multiple passes were made until 
quotas were filled. All students had the same chance to be assigned 
to either group on each pass. 

It should be noted here that the initial criteria for selecting the 
parent pool for random assignment had very definite l inl tat ions . 

There are many reasons why such procedures would lead to misclassif lo- 
cations. Some of the misclassif ied pupils were, in some cases, removed 
from the experimental program by school personnel. Such transfers 
would not occur in control groups, so pupil transfers could conceivably 
*have biased the original sample. For this reason, transfers were dis- 
couraged except in conspicuously necessary cases. 

Procedures were instituted to petmit checking the sources of sample 
attrition. 



Attrition 



Attrition was defined in terms of being present at the posttesting session 
as a member of the group to which the pupil was originally assigned. 
Slightly more than one- third of the original parent pool was lost due to 
attrition. Pupils were assigned to the parent pool prior to the opening 
of school. Pupils who moved during the summer or attended other schools 
could, therefore, not be included in the study. Absenteeism from the 
posttesting session, however, was found to be the greatest single source 
of attrition. Analyses of variance procedures were applied to pretest 
scores to estimate the effects of the attrition. ^ 

The effect of attrition upon pretest scores at the seventh— grade level 
seemed to be dependent upon the particular school and treatment group 
(experimental or control) . A significant three-way interaction was found 
among these three factors. At the sixth -grade level a two-way interaction 
was found between treatment groups and attrition. Attrition had little 
effect upon scores obtained by the control group. Students still in the 
experimental groups at the time of posttesting scored higher than the 
control group and the pupils who were no longer' in the experimental group. 
The difference in pretest scores was greatest between the two experimental 
groups (there versus not there) . 

Close inspection of the seventh-grade data revealed that biasing factors 
were generally working to favor the experimental group. The investigation 
of the effects of attrition upon the original sample, therefore, indicated 
biases which favored the experimental groups, especially at the sixth-grade 
level. There were no significant pretest differences between treatment 
groups prior to attrition. 



Testing 

As previously mentioned, the pretest data were constituted by scores on: 

1. Otis Short Form Intelligence Scale . 

2. Reading score on the California Test of Basic Skills , Level II . 

These tests were administered during the countywide testing in the spring 
of 1970. 

Posttesting was conducted during the first week in May 1971. It con- 
sisted of: 



1. Reading Vocabulary and R eading Comp r ehens i on sections of the 
California Test of Basic Skills , Level III . 

2. A 16— item attitude and opinion questionnaire created using 
suggestions from project teachers. 

Posttesting was carried out on a schoolwide basis. That is, all students 
'in both the control and the experimental groups in any school were tested 
simultaneously In one session. No provision was made for make-up exams 
for absentee students. 



Treatment 



The most important part of any research project of this type is’ an 
adequate description of the treatment. The reader must be told 
what actually happened to pupils in the treatment groups . Unfortunately , 
in the present case this is almost impossible. While the type of student 
and the equipment available were pretty much the same for all schools, 
the actual program that was implemented depended cn a number of additional 
factors. These were such things as: 

1. The physical size and arrangement of the project classes.- In 
some cases a regular classroom was used, while in other schools 
only a small back room in the library was available. 

2. The subject from which students were taken in order to attend 
project classes was not the same in each school. In some cases 
students were pulled from English, while in other instances 

_ students missed gym in order to be in Title I Reading. This 
may have had the paradoxical effect of lowering some students* 
motivation (because they are missing gym), while in effect 
' giving them a double dose of language arts. 

W 

Perhaps the most important factor to be considered in the context of this 
study is the manner in which the control group was treated. In many 
cases, instead of simply allowing control students to participate in a 
’'normal” school program, these pupils were also put into special reading 
programs. While this was undoubtedly done with the best of intentions 
(after all, these students were identified as reading below their 
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potential) , the actual effect may have been to blur the distinction 
between participation and nonparticipation in the Title I Reading Proj- 
ect. However, it can be argued that if some control pupils received 
similar services and results did not differ. Title I funds could be 
better allocated to providing more unique benefits . 

Project classes held a minimum of 12 and a max im u m of 15 students. 

Classes met once each day for one class period averaging 50 minutes. 

Teachers hired for this project were required to have special experience 
in teaching reading and had to have a thorough understanding of diagnosis 
and instruction in an individualized setting. Each student was tested by 
the project teacher, using the Stanford Diagnostic Reading Test , an instru- 
ment specifically designed to point out reading difficulties. Using the 
results of this test, a highly individualized program was worked out for 
each student using a multimedia approach. ■ ... 

The material available included Basal Readers, SFA Reading Kits, multi- 
level paperback books, programmed workbooks, and mechanical and electronic 
devices such as the Flashex and Hoffman Reader. After assessing a student f s 
area(s) of weakness, the teacher tried to program him into material appro- 
priate not only in difficulty, but also of a motivational and interest level 
*in keeping with his learning style. Following the guidelines set down by 
th*» county, an attempt was made to keep the orientation "phonic" in nature. 

In contrast to some projects of this type, the directors of Title I in 
Broward County did not create a long list of unrealizable and untes table 
goals. The program was designed simply to improve the reading ability cf 
participants. 



Results- 



CTBS Reading Test Scores 

The basic hypothesis of the study was that pupils who participated in the 
Title I program would score higher on the reading skills measured by the 
standardized test than the control group. Analyses of covariance procedures 
were used to test this hypothesis.* 

s, 

The basic research question to be answered by the statistical procedures 
was: did participation in the project enable students to score higher on 

the post tests than could be expected on the basis of their pretest scores? 
Stated as a problem in prediction, the hypothesis would be: does knowing 

both a person's pretest scores and whether he was in the experimental or 
control group make for significantly greater predictive accuracy in esti- 
mating his posttest scores than could be obtained from merely knowing his 
pretest scores but not his group membership? 



* A version of a MAN OVA program refined by Eliot Cramer, and a locally 
modified version of an SSP regression analysis program were used in 
making data a nal yses. 
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The hypotheses were made a little more complicated by the fact that 
knowledge of the school a pupil attended might also be important. The 
analyses which were run took, into account school differences, treatment 
classifications, and pretest results. 

It was found that posttest scores depended upon differences in pretest 
scores which "interacted" with school differences. In some schools 
pupils who made high pretest scores did better than pupils who made 
similarly high scores in other ■ schools . Low— scoring pupils in these 
same schools did just the opposite. In brief, some of the differences 
between schools depended upon how well students did on the pretests. 

Tn some schools persons with high scores on the pretest did better, but 
those with low scores worse than their counterparts in other schools. 

It was found that the treatment classification seemed to be independent 
of these interactions between schools and scores on the pretests. 

Within, each school scores of students in both the experimental and con- 
trol groups differed by similar amounts at all levels of pretest scores. 
Both groups followed the trend common to their particular school. 

These interactions did, however, make it necessary to consider differences 
"between treatment and control groups separately at each school. Had these 
interactions not been found, scores obtained by all experimental and 
control pupils could have been compared in one. overall analysis.* 

Comparisons broken down by individual schools are presented in Tables 3 
and A, The numbers presented in these tables represent differences in 
raw scores (the number of correctly answered items on each test) . The 
scores are "adjusted" in the sense that pupils in the experimental group 
would be expected to differ from pupils in the control group by this 
amount at all levels of the pretest scores. In other words, any two 
students with the same IQ and pretest reading scores would be predicted 
to differ by these amounts if one were in the experimental and the other 
in the control group. This difference would be unaffected by whether 
their pretest scores were high or low. 

Inspection of the tables indicates Only three instances of statistically 
significant d±f f erences. Two of the statistically significant differ- 
ences were in the same school at the sixth -grade level and were in favor 
of the control group. In School A, also on the sixth-grade level, the 
experimental group scored statistically significantly higher than the 
control group. None of the differences at the seventh-grade level reached 
the conventional .05 level of statistical significance. 

Forty comparisons were made, each with a probability of occurring by chance 
five- percent of the time. At these odds, little importance can be attached 
to the fact -that three significant results were obtained. 



* The details of the statistical procedures are briefly described in the 
Appendix for the benefit of technically oriented readers . 



Tab le 3 - 

Differences Between Experimental .and Control Group Means 
on Posttest Scores Adjusted for IQ and 
Pretest Reading Scores (6th Grade) 



Difference 
in Mean 
Scores 

In favor of 


School 1 


School 2 


School 3 


School 4 


School 5 


Voc. 


Comp .. 


Voc. 


Comp. 


Voc. 


Comp. 


Voc. 


Comp . 


Voc. 


Comp. 


2.37 


2.91 


0.59 


0.26 


1.00 


0.56 


3.52 


1.35 


1.52 


1.30 


Con- 

trol 


Con- 

trol 


Exp. i 


Con- 

trol 


Con- 

trol 


Exp. 


Exp. 


Exp. 


Exp. 


Con- 

trol 


Difference 
in Mean 
Scores 

In favor of 


P-.05] 


P-.032 P-.pC 


>1 




School 6 


School 7 


School 8 


School -9 ' 




Vocv j 


Comp. 


Voc. 


Comp. 


Voc. | Comp. 


Voc. 


Comp. 


0.51 


0.04 


1.40 


0.50 


0.485 0.93 


1.67 


0.04 


Con- 

trol 


Exp. 


Exp. 


Exp. 


Con- | 

trol 1 Exp . 


Con. 

trol 


Con. ; 
trol 



Table 4 



Differences Between Experimental and Control Group Means 
on Posttest Scores Adjusted for IQ and 
Pretest Reading Scores (7th Grade) 





School 1 


School- 2 


[ School 3 


School 4 ' 


School 5 


School 6 




Voc. i 


Ccrmp. 


Voc. 


Comp . 


[Voc. 


Comp. 


Voc. 


Comp . 


Voc. 


Ccrap. 


Voc. 


Comp , 


Difference 
in Mean 
Scores 


2.28 


0.71 


2.34 


2.44 ] 


i0.57 


1.13 


1.35 


1.23 


0.61 


0.48 


.1.45 


1.49 


In favor of 


Exp. 


Con- 

trol 


Con- 

trol 


~ Con- 
trol | 


jExp- 


Exp. 


Exp. 


Exp. 


Exp. 


Con- 

trol 


Con- 

trol 


Exp. 



Difference 
in Mean 
Scores 

In - favor oJ 


School 7 


School 8 


School 9 


School 10 


School 11 


Voc. 


Comp . 


Voc. 1 


Comp . 


Voc. 


Comp. 


Voc. 


Comp . 


Voc. 


Comp, 


0.19 


1.05 


0.73 


0.08 


0.01 


1.12 - 


1.18 


0.03 


0.71 


0.23 


’ Exp. 


Con- 

trol 


Exp. 


Con- 

trol 


Exp. 


Con- 

trol 


Con- 

trol 


Con- 

trol 


; Con- 
trol 


Con- 

trol 




In short, the Title I Reading Project seemed to have no effect upon the 
achievement of students as measured by the Reading section of the CTBS . 
Students in experimental classes did no better or worse than their 
counterparts in the control group. 

Attitude Questionnaire 

As was stated previously, the program was designed simply to improve the 
reading ability of underachieving sixth- and seventh-year students. 

There were no formal attitudinal goals. This does not mean, however, 
that some attitudinal change, sought or unsought, would not occur as a 
result of participation in the project. It is believed by many that 
underachievement in the reading area is a primary source of unhappy^ 
attitudes. For this reason, an attempt was made to assess students 
attitudes at' the end of the school year. 



B.oth experimental and control students answered a 16-item questionnaire. 
Table *5 gives the individual questions and percentage of pupils in both 
groups answering in each of the four response categories . Percents do not 
sum to 100 because a fifth "no response" category was not reported in these 
tables. Because of the nature of the data, the table is not broken down 
•by schools, but simply into sixth and seventh grades. A chi— square test 
was performed on each question to' see if there was a significant difference 
between control students and experimental students. Of the 32 comparisons, 
only one proved to be statistically significant. This "lonely soldier 
was Item 12 for the seventh grade. Since at least one item could be 
expected to be significant by chance alone, the probability is that the 
practical significance of this finding is nil. 

Apparently participation in the program did not affect the responses of 
students to the attitudinal questions. 

Table 5 

Percentage of Sixth- and Seventh-Grade Control and Experimental 
Students in All Schools Answering in Each of Four Response Categories 



A. Very, strong agreement 
' B. Some agreement 

C. Slight agreement 

D. Disagreement 

1. This year school has been very enjoyable. 

6th Grade 





A 


B 


C 


D 


Control 


A3. 5 


21.5 


19.8 


13.4 


Experimental 

• 


47.1 

1 


20.3 


20.3 


10.3 



7th Grade 



B 



30.7 


27.1 


20.4 


20.0 


34.4 


27.0 


18.8 


17.6 
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2. The teachers this year have been very good, 







6th Grade 






• 


7th Grade 






A 


B 


C 


D 




A 


B 


C 


D 


Control 


33.8 


22.0 


27.4 


15.0 




28.3 


22.4 


29.1 


18.5 


Experimental 


36.0 


28.3 


20.6 


12.2 




23.3 


26.3 


27.7 


20.8 


Homework is easier 


now than it 


was in the past. 












6th Grade 








7th Grade 




• 


A 


B 


C 


D 




A 


- B 


C 


I) 


Cpntrol 


24.1 


15.0 


22.5 


! 36.5 




20.4 


1 

17.3 


20.4 


40.5 


Experimental 


24.9 


14.1 


16.0 


| 42.5 




18.3 


13.1 


20.8 


45.9 


Reading books 


is a 


lot of 


fun. 


















6th Grade 








7th Grade 






A 


B 


C 


D 




A ' 


B 


C 


D 


Control 


45.1 


13.9 


24.1 


15.0 




31.8 


17.3 


20.8 


28.7 


Experimental 


54.0 


14.5 


16.0 


12.6 




39.4 


13.3 


21.0 


19.3 


Going as far as you 


can in school is important. 










6th Grade 








7th Grade 






A 


B 


C 


D 




A 


B 


C 


D 


Control 


79.0 


8.0 


7.5 


3.7 




84.2 


7.0 


3.9 


3.5 


Experimental 


82.7 


7.6 


3.4 


3.8 




87.8 . 


4.4 ; 


2.4 


3.4 


This year ny grades 


are better than they have been in the past# 




6th Grade 








7th Grade 






A 


B 


C 


D 




A 


B 


C 


D 


Control 


37.0 


19.8 


17.7 


23.6 




26.7 


18.8 1 


18.5 


34,6 


Experimental 


41.3 


18.3 


18.0 


19.9 




30.7 • 


22.0 J 


17.1 


28.2 
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7. If you are not doing well in one subject in school, it is a good idea to get 
special help. 
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8. The teachers in this school treat me fairly. 
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■9. Most of the time I would rather read a good book than watch television. 
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10* My teachers help me as much as they can. 
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. 6th Grade 
A B C D 
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I find it easier to read ny school books 



this year. 
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12. I am good friends with most of the kids In ny classes. 
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Significant of o( = 
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I like Language— Arts (Heading and English) 


i very 


much. 
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I like Science 


very much. 
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I like Mathematics very much. 














6th Grade 






7th Grade 






A 


B 


C 


D 


A 


B 


C 


IK 


Control 


45.1 


15.0 


17.2 


20.9 




39.7 


20.4 


16.1 


22.4 


Experimental 


47.8 


14.5 


! 14.5 


20.6 




33.7 


19.3 


18.3 


26.7 



16. 



I like Social Studies very much. 
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Discussion and Limitations 

Limitations 

True experimentation is almost impossible to conduct in a school system. 
The pressures and requirements of running an ongoing educational program 
make it difficult to meet the stringent demands of the researcher. As a 
result, most educational research is of the "field" variety. That is, 
one attempts to evaluate the program as it is , not as it would have been 
set up under ideal circumstances. Because of the lack of complete con- 
trol, an d because the treatment is not consistent from time to time, and 
place to place, results are often difficult to interpret. 

This study had three serious limitations: 

1. The programs were not monitored by research personnel to secure 
objective indices of whether the treatments offered to the two 
groups really differed in any substantial sense. 

2. The battery of tests was not complete enough. Further, the 
criterion measure was possibly too difficult a test for these 
groups . 

3. Provisions should have been made for testing students who missed 
the posttest sessions due to absenteeism or for other reasons. 



Discussion 

The most salient of the above limitations from the standpoint of this 
investigation is the one regarding the criterion test. The complete absence 
of a program effect leads one to suspect that some sort of artifact might 
be present in these data. Using too difficult a test could produce results 
like these, when a more sensitive test would reveal that considerable differ- 
ences did exist between the treatment groups- This lurking suspicion 
constitutes a major reason for the evaluative procedures which will be 
followed during the 1971-72 school year. 

At present, these findings appear to be quite unambiguous. The evidence 
collected in this document indicates that participation in the Title I 
Project had little effect upon the reading skills measured by the criterion 
test. Student attitudes also appeared to be unaffected by participation 
in the project. 

Unfortunately these findings were not available at the time the 1971—72 
Title I programs were being developed. It would seem in light of these 
findings and the previous evaluations of this project's predecessor,* 
that major changes should be made in planning Title I programs for the 
1972-73 school year. - 

* Title I funds at this grade level had been previously used to support 
Project GAIN (Gearing Academics t° Individual Needs). See report 
Nos. 1, 19, 32i and. 34 for evaluations of this project. 
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The Research Department has a contractual obligation to provide an evalu- 
ation of this project for the 1971-72 school year. This obligation will 
be fulfilled by retesting the sample groups whose results are reported in 
this document . Retesting will be done early in the Fall of the 1971-72 
school year. This retesting will serve the following functions: 

1. It will permit assessing the effects of this program in terms of 
factors such as retention of skills. 

2. It will permit testing the hypothesis that an inappropriate test 
was used in the current study. Tests suggested by the project 
director will be used in the follow-up study. 

3. It will provide evaluative feedback in time to permit appropriate 
administrative action for the following year. 

In view of the above plans , it would seem premature to draw conclusions or 
make recommendations prior to the completion of the projected follow-up 
study. 

From a subjective standpoint,, Research personnel associated with this 
project felt that it had a better chance for success than the project it 
supplanted. Goals and objectives were more narrowly defined, policies and 
procedures were more uniform, and the overall coordination and supervision 
of the project seemed to be more effective. The results of this evaluation 
are disappointing inasmuch as there seemed to be good reason to expect the 
project to be more successful than is indicated by the present evidence. 

It is to be hoped that the follow-up study will provide evidence which will 
point in a more positive direction. It should be pointed out, however, 
that the present study had sufficient merit to warrant regarding its find- 
ings with serious concern until such time as they are confirmed or refuted 
by the follow-up evaluation. 



\ 
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APPENDIX 



Statistical Procedures 

A two-way (treatment by school) Manova run was made using pretest IQ and 
reading achievement as covariates. The dependent measures were posttest 
scores on Reading Vocabulary and Comprehension. The tests for the 
equality of the regression coefficients were statistically significant. 
The data were rerun separately by schools in another regression analysis 
program and it was found that the regression planes for treatment groups 
could be regarded as parallel within schools. Finally, the data were 
rerun in the Manova program with the same factors and variables. This 
time, however> a "pseudo" nested model was used (treatments nested within 
school) . This resulted in lumping the variance due to treatment and 
interaction into a "pseudo" nested effect. Variance solely due to the 
school factor was partitioned as before. Tests of the significance of 
the differences between treatment groups within each school were provided 
by application of these procedures. In other words, the nested effect 
was partitioned by treatment groups within schools. Since a common 
regression plane could not really be fitted for all schools, separate 
tests of differences between treatment means within each school were also 
made as part of the regression procedures. Both the regression analyses 
and the Manova runs produced similar results. The contrasts reported in 
Tables 3 and 4 were taken from the Manova output. Had nonhomogeneity of 
regression mattered greatly, the constants based upon the regression 
planes unique to each school would have been reported instead of the 
Manova contrasts. 
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